Video coding decoding apparatus

ABSTRACT

This invention relates to a video coding decoding apparatus for effectively refreshing a video and for recovering degradation of a video caused by code error, in a short time. A video coding decoding apparatus according to the present invention comprises area shape detection (12) for dividing the picture in the single frame into a motion compensation area and the remaining area and then detecting a shape of the motion compensation area by distinguishing between the motion compensation area and the remaining area, and motion vector estimation (14) for estimating a resultant motion vector of the overall motion compensation area from a motion vector of each block in the motion compensation area. A refreshing operation is carried out from a starting point of the estimated resultant motion vector towards the direction of the estimated resultant motion vector in a motion compensation area. When a code error has occurred, code error block number informing circuit (13) informs the number of the block where the code error has occurred. The starting point of the refreshing operation is a block which, among many other adjacent blocks to the informed block, is located in an opposite direction with respect to the estimated motion vector. For this reason, a video can be refreshed effectively and degradation of a video caused by code error, can be recovered in a short time.

TECHNICAL FIELD

This invention relates to a video coding decoding apparatus which iscapable of effectively refreshing a picture and recovering degradationof a picture caused by code error, in a short time.

BACKGROUND ART

Heretofore, there is known a system, as a technique for coding a video,in which a picture in a single frame is divided into blocks of M×N(where M and N are natural numbers, respectively). In the technique,data showing the frame, data showing the number of each block and dataof a picture belonging to the blocks are coded in a hierarchicalstructure, and those data are coded by means of a two-dimensionaldiscrete cosine transform (DCT) and a Huffman code (ITU-T RecommendationH. 261). If, at that time, a motion on each block in the continuousinter-frames is estimated, data can be coded in an efficient manner bycompensating the motion from a reference data and without coding all ofthe picture data.

Here, in the example shown in FIG. 1, the block numbers are assignedsuch that a number assigned to a block located on a left upper end ofthe explanatory view is "1" and the numbers are gradually sequentiallyincreased toward the right of the same horizontal row. When a blocklocated on the right extremity of the row is reached, a new row isstarted and a new block number is assigned to each block on this new rowfrom left to right in the same manner as the first row. It should benoted that FIG. 1 shows only one example showing a way for assigningblock numbers and any other suitable way for assigning block numbers maybe employed in accordance with necessity. In a video coding decodingapparatus in which the above-mentioned coding system is employed, arefreshing operation is carried out in a sequential order of the blocknumbers for each frame (see arrows of the broken lines). Presuming thatthe number of a block in which a refreshing operation is carried out inan i-th frame is "1", the number of another block in which a refreshingoperation is carried out in the subsequent (i+1)-th frame is "2". Itshould be noted that such a refreshing operation is carried out on apremise that a communication line, in which a transmission error isnegligible, is employed and no consideration is given to a possibledegradation caused by code error.

However, in the event that a code error has occurred in an area where amotion compensation is being carried out and this particular area is notto be refreshed in the subsequent frame, degradation of a picture isaccumulated without being recovered even when no code error has occurredin the above-mentioned subsequent frame. Consequently, in accordancewith the conventional refreshing operation, once a picture degradationoccurs, it takes about several dozens of seconds to recover the picturefrom degradation in the worst case.

DISCLOSURE OF INVENTION

The present invention has been accomplished in view of theabove-mentioned problems. It is, therefore, an object of the inventionto provide a video coding decoding apparatus in which a refreshingoperation for recovering degradation of a picture caused by code errorcan be carried out efficiently.

In order to achieve the above object, there is provided a video codingdecoding apparatus in which a picture in a single frame is divided intoblocks of M×N (where M and N are natural numbers) and in which aninterframe motion compensation prediction is carried out, the videocoding decoding apparatus comprising:

area shape detecting means for dividing the picture in a single frameinto a motion compensation area and the remaining area and thendetecting a shape of the motion compensation area by distinguishingbetween the motion compensation area and the remaining area;

motion vector estimating means for estimating a motion vector of theoverall motion compensation area from a motion vector in the motioncompensation area; and

refreshing means for carrying out a refreshing operation from a startingpoint of the estimated motion vector toward the motion vector, in themotion compensation area.

According to the present invention, a motion compensation area in avideo in a single frame is detected by the area shape detecting means, amotion vector (moving direction) of the overall motion compensating areais estimated from motion vector of each block in the motion compensationarea by the motion vector estimating means, and a refreshing operationis carried out from a starting point of the estimated motion vectortoward the estimated motion vector (moving direction). Consequently,since the picture in the motion vector compensation area is graduallyrefreshed toward the motion vector, it becomes natural. Further, sinceno refreshing operation is carried out with respect to a static areadisposed in an opposite direction to the motion vector, a refreshingoperation can be carried out in an efficient manner.

Furthermore, according to the present invention, the apparatus furthercomprises code error block number (where code error block number is anumber of the block to which code error occurred) informing means forinforming a number of the block where a code error has occurred, and ablock which, among many other adjacent blocks to that block which isinformed by the block number informing means, is located in an oppositedirection with respect to the estimated motion vector, is served as thestarting point for the refreshing operation. Accordingly, when a codeerror has occurred to a block in the motion compensating area, arefreshing operation is carried out from this way of the block towardthe estimated motion vector. Consequently, degradation of a picturecaused by code error can be recovered rapidly.

Moreover, according to the present invention, the apparatus furthercomprises refreshing range control means for limiting the refreshingrange to a range of blocks belonging to the motion compensation area.Accordingly, since the range of a refreshing operation is limited onlyto the motion compensation area. Consequently, a refreshing operationcan be omitted with respect to those blocks which are located outside ofthe motion compensation area, or those blocks disposed in a static area.For this reason, a refreshing operation can be carried out moreefficiently.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an explanatory view for explaining a conventional refreshingprocedure;

FIG. 2 is a block diagram showing a construction of a video codingdecoding apparatus according to a first embodiment of the presentinvention;

FIG. 3 is a block diagram showing a construction of an area shapedetecting means for the video coding decoding apparatus according to thefirst embodiment of the present invention;

FIG. 4 is a block diagram showing a construction of a motion vectorestimating means for the video coding decoding apparatus according tothe first embodiment of the present invention;

FIG. 5 is an explanatory view showing a refreshing procedure for thevideo coding decoding apparatus according to the first embodiment of thepresent invention; and

FIG. 6 is an explanatory view showing a refreshing procedure for a videocoding decoding apparatus according to the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

A video coding decoding apparatus according to a first embodiment of thepresent invention will be described hereinafter with reference to thedrawings. FIG. 2 is a block diagram showing a construction of thisembodiment. In this illustration, reference numeral 11 denotes a maincoding unit for carrying out a two-dimensional discrete cosine transform(DCT) with respect to a prediction error occurrable when a motioninter-frame prediction is carried out in blocks having a size of n×n (nis a natural number) and coding a transfer coefficient thereof. It ispresumed here that a picture in a single frame is constituted of blocksof M×N (see FIGS. 1, 5 and 6). Also, this main coding unit 11 isoperated to refresh blocks of those block numbers received, as laterdescribed.

Reference numeral 12 denotes an area shape detecting means for dividinga picture in a single frame into a range of a motion compensation areaand the remaining area and then distinguishing a shape of the motioncompensation area by distinguishing the motion compensation area fromthe remaining area. A construction of the area shape detecting means isshown in FIG. 3. In this illustration, a comparator 12¹ compares apre-established threshold MVth with a magnitude of a motion vector MV ofeach block supplied by the main coding unit 11 and extracts a block orblocks having a motion vector MV of a magnitude equal to or more thanthe pre-established threshold MVth (which block or blocks are to besubjected to motion compensation).

This will be described in more detail. In the case where a magnitude ofthe motion vector is coded in two-dimension consisting of a horizontaldirection and a vertical direction, it is presumed that a motion vectorhaving a block number of "n" can be expressed as follows.

    MV.sub.n =(x.sub.n,y.sub.n)

If the magnitude of this motion vector is represented by MVd(n), thesame can be expressed as follows. ##EQU1## This MVd(n) is obtained foreach block and compared with the threshold MVth, thereby to obtain ablock or blocks to be subjected to motion compensation.

For making a judgment, a microscopic motion is basically disregarded anda macroscopic motion such as, for example, a motion of a figure movingin front of a fixed background, is detected as a moving direction of amotion vector of the overall motion area.

Then, the block combining means 12₂ combines the blocks extracted by thecomparator 12₁ into a single area (unitary area) and outputs data aboutthe area shape together with the motion vector MV of each block.

Referring back to FIG. 2, reference numeral 13 denotes a code errorblock number informing means for outputting the numbers of blocks whichcould not be decoded due to occurrence of a code error. The code errorblock number informing means is disposed on a receiving side. It shouldbe noted that this informative data may be transmitted by multiplexingthe related data on a coded data.

Reference numeral 14 denotes a motion vector estimating means. As shownin FIG. 4, as well as elsewhere, the motion vector estimating means 14combines various motion vectors in a motion compensation area detectedby the area shape detecting means 12 and estimates a motion vector ofthe specific area as a whole. Upon input of an area shape data by thearea shape detecting means 12 and a code error block number by the codeerror block number informing means 13, the motion vector estimatingmeans 14 makes a judgment as to which area the block specified by theabove number belongs to. If the specific area belongs to the motioncompensation area, the motion vector estimating means 14 outputs thenumber of that block which, among error-occurred blocks in the motioncompensation area, is located in the direction of a starting point inthe sense of a motion vector (moving direction). In contrast, if theblock, in which a code error has occurred, is outside the motioncompensation range or if the block belongs to a static area, the motionvector estimating means 14 outputs the number of the block in which thecode error has occurred.

Next, operation of the above-mentioned embodiment will be described.First, there will be described operation when no code error hasoccurred. The motion vector MV of each block is extracted from areceived video signal by the main coding unit 11. The motion vector MVof each block is compared with the threshold MVth by the comparator 12₁of the area shape detecting means 12. Then, the blocks having a motionvector MV equal to or more than the threshold are combined by the blockcombining means 12₂. By doing this, a range of blocks having a motionvector equal to or more than the threshold MVth, i.e., a motion area tobe subjected to motion compensation, is detected. Subsequently, themotion vectors of the individual blocks are composed. By doing this, themotion vector of the overall motion compensation area is estimated bythe motion vector estimating means 14.

Here, as is shown in FIG. 5, the motion area to be subjected to motioncompensation is widely spread and the estimated motion vector is likelyto be a direction indicated by a bold arrow. In this case, the motionvector estimating means 14 supplies the block numbers one after anotherto the main coding unit 11 for each frame in a direction indicated by abroken line arrow so that a refreshing operation can be carried out fromthe block located in the motion area but in the most opposite directionwith respect to the estimated motion vector (i.e., block located at thestarting point of the estimated motion vector (namely, block indicatedby START in FIG. 5)) toward the estimated motion vector. By this, themain coding unit 11 carries out a refreshing operation for each frame,one after another, with respect to the blocks corresponding to the blocknumbers supplied by the motion vector estimating means 14.

As a consequence, since the picture in the motion area is graduallyrefreshed from the starting point, it becomes natural. Further, since norefreshing operation is carried out with respect to the block or blockslocated in an opposite direction to the motion vector and in the staticarea, a refreshing operation can be carried out in an efficient manner.

Next, there will be described an operation when a code error hasoccurred to the received coded data. The numbers of the blocks, in whicha code error has occurred but in which decoding was failed to carry out,are informed by the code error block number informing means 13. Then,the motion vector estimating means 14 makes a judgment as to which areathe blocks of the informed block numbers belong to. When the blocks ofthe informed block numbers are located in the motion compensation area,the motion vector estimating means 14 outputs the number of the blockwhich, among many other adjacent blocks adjacent to the informed blocks,is situated in an opposite (starting point) direction with respect tothe motion vector, as the starting point for a refreshing operation. Theblocks to be subsequently subjected to refreshing operation are the sameas the above-mentioned case where no code error has occurred, in therespect that the blocks to be subsequently subjected to refreshingoperation are to be refreshed in the motion vector (moving direction).

On the other hand, when the blocks, in which a code error has occurred,are outside the motion compensation area or belong to the static area,the motion vector estimating means 14 outputs the numbers of the blocksso that the very blocks of the informed block numbers are refreshed.Thereafter, the motion vector estimating means 14 successively outputsthe block numbers so that refreshing operation is continuously carriedout in the direction of the motion vector.

As seen, when a block, in which a code error has occurred, belongs tothe motion compensation area, a refreshing operation is carried out fromthis way of the block in the estimated motion direction. When the block,in which a code error has occurred, is outside the motion compensationarea or belongs to the static area, this block is immediately refreshed.Accordingly, there can be obtained a rapid recovery of a video fromdegradation caused by code error.

A second embodiment of the present invention will now be described. Inthe above-mentioned first embodiment, the refreshing operation iscarried out only in the estimated motion vector. The second embodimentis the same as the first embodiment in the respect that the startingpoint of the estimated motion vector is served as a starting point forthe refreshing operation. The second embodiment is characterized in thatthe refreshing operation to be carried out in the estimated motionvector is limited to those blocks including or belonging to the motioncompensation area. This is made possible for the motion vectorestimating means 14 by outputting the numbers of blocks located in themotion compensation area after it outputs the block number serving as astarting point for the refreshing operation. At that time, the areashape data detected by the area shape detecting means 12 is utilized.

In this case, a refreshing operation is carried out in the sequentialorder as indicated by the broken lines of FIG. 6. Owing to thisarrangement, those blocks in the static area are not subjected torefreshing operation and only those blocks belonging to the motion areaare subjected to refreshing operation. Accordingly, a refreshingoperation can be carried out in a very efficient manner. It should benoted that an arrangement is also possible in this second embodimentthat in case of an occurrence of a code error, when the code error hasoccurred in the motion compensation area, the number of the block which,among many other blocks adjacent to the blocks where an code error hasoccurred and which is located in the direction of a starting point inthe sense of a motion vector (moving direction), is output and when theblocks, in which a code error has occurred, are outside the motioncompensation area or belong to the static area, the number of theerror-occurred blocks are output.

INDUSTRIAL APPLICABILITY

The present invention is suited to be applied to a case where data of avideo having a comparatively gentle motion is transmitted in a two-waycommunications by a code error inherent communication system such as,for example, a digital mobile communication system. In the future, thepresent invention is believed to be applicable to a mobile videotelephone, a video conference system, and the like.

We claim:
 1. A video coding/decoding apparatus of which a picture in asingle frame is divided into blocks of M×N (where M and N are naturalnumbers) and in which an interframe motion compensation prediction iscarried out, said video coding decoding apparatus comprising:area shapedetecting means for dividing the picture of a single frame into a motioncompensation area and the remaining area and then detecting a shape ofsaid motion compensation area by distinguishing between said motioncompensation area and said the remaining area; motion vector estimatingmeans for estimating a resultant motion vector of the overall motioncompensation area from a motion vector of each block in said motioncompensation area; and refreshing means for carrying out a refreshingoperation from a starting point of said estimated resultant motionvector towards the direction of said estimated resultant motion vector,in said motion compensation area.
 2. A video coding/decoding apparatusaccording to claim 1, further comprising a refreshing range controlmeans for limiting a range of said refreshing operation to a range ofsaid motion compensation area.
 3. A video coding/decoding apparatusaccording to claim 1, wherein:said motion vector estimating meansestimates said resultant vector of said overall motion compensation areaby combining motion vectors of the respective blocks belonging to saidmotion compensation area, and said refreshing means refreshes each ofsaid blocks belonging to said motion compensation area for each framefrom a starting point of said estimated resultant motion vector towardsthe direction of said estimated resultant motion vector.
 4. A videocoding/decoding apparatus according to claim 1, further comprising:codeerror block number informing means for informing a number of a block inwhich a code error has occurred; a block which, among many otheradjacent blocks to that block which is informed by said block numberinforming means, located in the opposite direction with respect to saidestimated resultant motion vector, being served as a starting point forsaid refreshing operation.
 5. A video coding/decoding apparatusaccording to claim 4, further comprising:refreshing range control meansfor limiting a range of the refreshing operation to a range of blockswhich belong to said motion compensation area.